Synthetic Minority Oversampling Technique Research Articles

Deep convolutional neural network based synthetic minority over sampling technique: a forfending model for fraudulent credit card transactions in financial institution

Today, fraudulent credit card transaction are committed by unauthorized individuals and organization exploring methods such as phishing and social engineering fraud. Several researchers proposed Machine Learning (ML) techniques to serve as solutions to credit card fraud detection. However, ML suffers imbalance class distribution, high dimensionality and sparsity problem. These concerns make it extremely difficult in credit card fraud detection. Therefore, this study proposes Deep Convolutional Neural Network (DCNN) with Synthetic Minority Oversampling Techniques (SMOTE) to serve as an ideal solution. Data augmentation method of SMOTE oversampling techniques is incorporated into the DCNN that solve the class imbalance trial and in improving the performance. Datasets with 284807 records with 31 features gotten via Kaggle repository is used. Implementation is carried out on Google Colab containing cloud-based platform embedding Jupyter notebook environment with Graphical processing unit (GPUs). Two experiment were carried out where the first experiment is used to determine suitable models among baseline techniques: Logistics Regression (LR), Random Forest (RF), Isolation forest, and single DL model of Multiple layer perceptron (MLP). The baseline models yielded an overfitting accuracy score, with recall, specificity, precision, and F1-score all presenting 1.00% respectively. This outcome is not sufficient in establishing findings on imbalance data distribution as its biased. This led to the construction of a new ML model absorbing Light Gradient Boosting Machine (LGBM), with Artificial Neural Network (ANN) and proposed DCNN+SMOTE for the second experiment phase alongside baseline models. Experimental results via simulation show the proposed DCNN+SMOTE yielded overwhelming superclass performance across board displaying 1.00% results respectively better than baseline models. It Error Rate (ER) and Null Error Rate (NER) is 0.00% distinctly. While, the False Positive Rate (FPR) yields 0.001% result lesser than the baseline models.

An Improved and Optimized Random Forest Based Approach to Predict the Software Faults

Effective software fault prediction is crucial for minimizing errors during software development and preventing subsequent failures. This research introduces an enhanced Random Forest-based approach for predicting software faults, specifically focusing on the NASA JM1 dataset. The dataset comprises 21 software metrics indicating the presence or absence of faults in a module, and it is utilized to evaluate the proposed approach. The study delves into the intricacies of the NASA dataset, detailing the cleaning process and addressing class imbalance through Synthetic Minority Over-sampling Technique (SMOTE). The core of our approach involves the implementation and fine-tuning of the Random Forest classifier, with a specific focus on optimizing hyperparameters to enhance predictive accuracy. In comparative evaluations with standard machine learning models, our proposed approach demonstrated superior performance, achieving an accuracy of 82.96% and an F1 score of 89.53%. Notably, we emphasize the significance of software defects and their potential to cause failures and crashes during software development, leading to substantial organizational losses. The paper provides a comprehensive examination of different aspects of the machine learning model, offering detailed insights, examples, and illustrative figures to enhance the understanding of our proposed approach.

A comprehensive study on machine learning models combining with oversampling for bronchopulmonary dysplasia-associated pulmonary hypertension in very preterm infants

BackgroundBronchopulmonary dysplasia-associated pulmonary hypertension (BPD-PH) remains a devastating clinical complication seriously affecting the therapeutic outcome of preterm infants. Hence, early prevention and timely diagnosis prior to pathological change is the key to reducing morbidity and improving prognosis. Our primary objective is to utilize machine learning techniques to build predictive models that could accurately identify BPD infants at risk of developing PH.MethodsThe data utilized in this study were collected from neonatology departments of four tertiary-level hospitals in China. To address the issue of imbalanced data, oversampling algorithms synthetic minority over-sampling technique (SMOTE) was applied to improve the model.ResultsSeven hundred sixty one clinical records were collected in our study. Following data pre-processing and feature selection, 5 of the 46 features were used to build models, including duration of invasive respiratory support (day), the severity of BPD, ventilator-associated pneumonia, pulmonary hemorrhage, and early-onset PH. Four machine learning models were applied to predictive learning, and after comprehensive selection a model was ultimately selected. The model achieved 93.8% sensitivity, 85.0% accuracy, and 0.933 AUC. A score of the logistic regression formula greater than 0 was identified as a warning sign of BPD-PH.ConclusionsWe comprehensively compared different machine learning models and ultimately obtained a good prognosis model which was sufficient to support pediatric clinicians to make early diagnosis and formulate a better treatment plan for pediatric patients with BPD-PH.

A multichannel analysis of imbalanced computed tomography data for lung cancer classification

Lung cancer holds the highest fatality rate among cancers, emphasizing the importance of early detection. Computer algorithms have gained prominence across various domains, including lung cancer diagnosis. These algorithms assist specialists, especially in medical imaging, yet current efforts lack comprehensive CT data analysis; especially in handling imbalanced datasets and fully exploiting spatial information. The lack of spatial analysis hinders the ability to identify subtle variations in texture and structure that are crucial for detecting lung cancer early and accurately. Therefore, this study uses a multichannel analysis of computed tomography (CT) images and deep learning-based ensemble learning (MC-ECNN) to find lung cancer even when the data is not balanced. Firstly, the data imbalance issue is tackled through the synthetic minority oversampling technique (SOMTE); afterwards, a multi-channel analysis of the data is performed to explore a distinct set of abstract features. Lastly, a deep ensemble learning method is used to classify the extracted distinct abstract feature set into the appropriate classes. The proposed method uses the discrete Fast Fourier transform (DFFT) and discrete cosine transform (DCT), along with the actual CT scans, for the multi-channel analysis of the data in different domains. The proposed model yielded 99.60% test accuracy on unseen data, which is at least 3% better than the other state-of-the-art studies considered for the comparison. In addition to the classification accuracy, the efficacy of the proposed model has also been justified through precision, recall, F1-score, support value, and misclassification rate.

A Novel Data Augmentation Method for Radiomics Analysis Using Image Perturbations.

Radiomics extracts hundreds of features from medical images to quantitively characterize a region of interest (ROI). When applying radiomics, imbalanced or small dataset issues are commonly addressed using under or over-sampling, the latter being applied directly to the extracted features. Aim of this study is to propose a novel balancing and data augmentation technique by applying perturbations (erosion, dilation, contour randomization) to the ROI in cardiac computed tomography images. From the perturbed ROIs, radiomic features are extracted, thus creating additional samples. This approach was tested addressing the clinical problem of distinguishing cardiac amyloidosis (CA) from aortic stenosis (AS) and hypertrophic cardiomyopathy (HCM). Twenty-one CA, thirty-two AS and twenty-one HCM patients were included in the study. From each original and perturbed ROI, 107 radiomic features were extracted. The CA-AS dataset was balanced using the perturbation-based method along with random over-sampling, adaptive synthetic (ADASYN) and the synthetic minority oversampling technique (SMOTE). The same methods were tested to perform data augmentation dealing with CA and HCM. Features were submitted to robustness, redundancy, and relevance analysis testing five feature selection methods (p-value, least absolute shrinkage and selection operator (LASSO), semi-supervised LASSO, principal component analysis (PCA), semi-supervised PCA). Support vector machine performed the classification tasks, and its performance were evaluated by means of a 10-fold cross-validation. The perturbation-based approach provided the best performances in terms of f1 score and balanced accuracy in both CA-AS (f1 score: 80%, AUC: 0.91) and CA-HCM (f1 score: 86%, AUC: 0.92) classifications. These results suggest that ROI perturbations represent a powerful approach to address both data balancing and augmentation issues.

Binding Activity Classification of Anti-SARS-CoV-2 Molecules using Deep Learning Across Multiple Assays

The coronavirus disease-2019 (COVID-19) pandemic, caused by severe acute respiratory syndrome-coronavirus-2 (SARS-CoV-2), has urgently necessitated effective therapeutic solutions, with a focus on rapidly identifying and classifying potential small-molecule drugs. Given traditional methods’ labor-intensive and time-consuming nature, deep learning has emerged as an essential tool for efficiently processing and extracting insights from complex biological data. To utilize deep learning techniques, particularly deep neural networks (DNN) enhanced with the synthetic minority oversampling technique (SMOTE), to enhance the classification of binding activities in anti-SARS-CoV-2 molecules across various bioassays. We used 11 bioassay datasets covering various SARS-CoV-2 interactions and inhibitory mechanisms. These assays ranged from spike-ACE2 protein-protein interaction to ACE2 enzymatic activity and 3CL enzymatic activity. To address the prevalent class imbalance in these datasets, the SMOTE technique was employed to generate new samples for the minority class. In our model-building approach, we divided the dataset into 80% training and 20% test sets, reserving 10% of the training set for validation. Our approach involved employing a DNN that integrates ReLU and sigmoid activation functions, incorporates batch normalization, and uses Adam optimization. The hyperparameters and architecture of the DNN were optimized through various tests on layers, minibatch sizes, epoch sizes, and learning rates. A 40% dropout rate was incorporated to mitigate overfitting. For model evaluation, we computed performance metrics, such as balanced accuracy (BACC), precision, recall, F1 score, Matthews’ correlation coefficient (MCC), and area under the curve (AUC). The performance of the DNN across 11 bioassay test sets revealed varying outcomes, significantly influenced by the ratios of active-to-inactive compounds. Assays, such as AlphaLISA and CoV-PPE, demonstrated robust performance across various metrics, including BACC, precision, recall, and AUC, when configured with more balanced ratios (1:3 and 1:1, respectively). This suggests the effective identification of active compounds in both cases. In contrast, assays with higher imbalance ratios, such as 3CL (1:38) and cytopathic effect (1:15), demonstrated higher recall but lower precision, highlighting challenges in accurately identifying active compounds among numerous inactive compounds. However, even in these challenging settings, the model achieved favorable BACC and recall scores. Overall, the DNN model generally performed well, as indicated by the BACC, MCC, and AUC values, especially when considering the degree of dataset imbalance in each assay. This study demonstrates the significant impact of deep learning, particularly DNN models enhanced with SMOTE, in improving the identification of active compounds in bioassay datasets for COVID-19 drug discovery, outperforming traditional machine learning models. Furthermore, this study highlights the efficacy of advanced computational techniques in addressing high-throughput screening data imbalances.

Sampling-based novel heterogeneous multi-layer stacking ensemble method for telecom customer churn prediction

In recent times, customer churn has become one of the most significant issues in business-oriented sectors with telecommunication being no exception. Maintaining current customers is particularly valuable due to the high degree of rivalry among telecommunication companies and the costs of acquiring new ones. The early prediction of churned customers may help telecommunication companies to identify the causes of churn and design industrial tactics to address or mitigate the churn problem. Controlling customer churn by developing efficient and reliable customer churn prediction (CCP) solutions is essential to achieving this objective. Findings from existing CCP studies have shown that numerous methods, such as rule-based and machine-learning (ML) mechanisms, have been devised to solve the CCP problem. Nonetheless, the problems of adaptability and the resilience of rule-based CCP solutions are its major weaknesses, and the skewed pattern of churn datasets (class imbalance) is detrimental to the prediction performances of conventional ML models in CCP. Hence, this research developed a robust heterogeneous multi-layer stacking ensemble method (HMSE) for effective CCP. Specifically, in the HMSE method, the prediction prowess of five ML classifiers (Random Forest (RF), Bayesian network (BN), Support Vector Machine (SVM), K-Nearest Neighbour (KNN), and Repeated Incremental Pruning to Produce Error Reduction (RIPPER)) with distinct computational characteristics are ensembled based on stacking and the resulting model is further enhanced using a forest penalizing attribute (FPA) model. The synthetic minority oversampling technique (SMOTE) is integrated with the proposed HMSE to balance the skewed class label present in the original experimental datasets. Extensive tests were carried out to determine the effectiveness of the proposed HMSE and S-HMSE on standard telecom CCP datasets. Observed findings from the experimental results showed that HMSE and S-HMSE can effectively predict churners even with the class imbalance (skewed datasets) problem. In addition, comparison studies demonstrated that the suggested S-HMSE offered improved prediction performance and optimum solutions for CCP in the telecom sector in comparison with baseline classifiers, homogeneous ensemble methods, and current CCP approaches.

Stone decision engine accurately predicts stone removal and treatment complications for shock wave lithotripsy and laser ureterorenoscopy patients.

Kidney stones form when mineral salts crystallize in the urinary tract. While most stones exit the body in the urine stream, some can block the ureteropelvic junction or ureters, leading to severe lower back pain, blood in the urine, vomiting, and painful urination. Imaging technologies, such as X-rays or ureterorenoscopy (URS), are typically used to detect kidney stones. Subsequently, these stones are fragmented into smaller pieces using shock wave lithotripsy (SWL) or laser URS. Both treatments yield subtly different patient outcomes. To predict successful stone removal and complication outcomes, Artificial Neural Network models were trained on 15,126 SWL and 2,116 URS patient records. These records include patient metrics like Body Mass Index and age, as well as treatment outcomes obtained using various medical instruments and healthcare professionals. Due to the low number of outcome failures in the data (e.g., treatment complications), Nearest Neighbor and Synthetic Minority Oversampling Technique (SMOTE) models were implemented to improve prediction accuracies. To reduce noise in the predictions, ensemble modeling was employed. The average prediction accuracies based on Confusion Matrices for SWL stone removal and treatment complications were 84.8% and 95.0%, respectively, while those for URS were 89.0% and 92.2%, respectively. The average prediction accuracies for SWL based on Area-Under-the-Curve were 74.7% and 62.9%, respectively, while those for URS were 77.2% and 78.9%, respectively. Taken together, the approach yielded moderate to high accurate predictions, regardless of treatment or outcome. These models were incorporated into a Stone Decision Engine web application (http://peteranoble.com/webapps.html) that suggests the best interventions to healthcare providers based on individual patient metrics.

Evaluation of performance enhancement in Ethereum fraud detection using oversampling techniques

With the growing popularity of cryptocurrencies and their decentralized nature, the risk of fraudulent activities within these ecosystems has become a pressing concern. This research paper focuses on Ethereum fraud detection using a dataset specifically curated for this purpose. The methodology encompasses essential steps, including data cleaning, correlation analysis, data splitting, and exploratory data analysis to understand the data characteristics. Subsequently, self-optimized machine learning models are trained with the Pycaret library while addressing the class imbalance using SMOTE (Synthetic Minority Over Sampling Technique), ADA-SYN (Adaptive Synthetic Algorithm), and K-Means-SMOTE (Support Vector Machine Assisted Synthetic Minority Over Sampling Technique) techniques. The performance of the various models is evaluated on test and validation datasets using metrics such as accuracy, precision, recall, and AUC. The study reveals that the ensemble models, particularly CATBoost (Categorical Boost) and LGBM (Light Gradient Boost Method), show exceptional efficiency, with accuracy ranging from 97% to 98.42% after oversampling. Moreover, these models exhibit higher F1 scores and AUC (region of convergence) values, indicating their potential to detect fraud effectively. The validation metrics also lie in the same range, demonstrating that the models do not suffer from over-fitting. The experiment demonstrates the promise of ensemble models in Ethereum fraud detection, paving the way for deploying robust fraud detection systems in crypto-currency ecosystems. The results show that the K- Means SMOTE oversampling technique has the highest classification accuracy levels of 98.42% with an AUC of 99.82%. These methods provide a 50-50 class balance for Ethereum fraud detection.

Fast Naïve Bayes classifiers for COVID-19 news in social networks

The growth of fake news has emerged as a substantial societal concern, particularly in the context of the COVID-19 pandemic. Fake news can lead to unwarranted panic, misinformed decisions, and a general state of confusion among the public. Existing methods to detect and filter out fake news have accuracy, speed, and data distribution limitations. This study explores a fast and reliable approach based on Naïve Bayes algorithms for fake news detection on COVID-19 news in social networks. The study used a dataset of 10,700 tweets and applied text pre-processing, term-weighting, document frequency thresholding (DFT), and synthetic minority oversampling techniques (SMOTE) to prepare the data for classification. The study assessed the performance and runtime of four models: gradient boosting (GDBT), decision tree (DT), multinomial Naïve Bayes (MNB), and complement Naïve Bayes (CNB). The testing results showed that the CNB model reaches the highest accuracy, precision, recall, and F1-score of approximately 92% each, with the shortest runtime of 0.55 seconds. This study highlights the potential of the CNB model as an effective tool for detecting online fake news about COVID-19, given its superior performance and rapid processing time.

Machine-learning-aided regional post-seismic usability prediction of buildings: 2016-2017 Central Italy earthquakes

This study addresses the development of machine learning models for predicting the post-seismic buildings usability within regions prone to frequent earthquakes. The analysis leverages on field data from 2016-2017 Central Italy earthquakes. Several machine learning techniques are employed for this task, namely K-nearest neighbors, linear support vector machine, radial basis function support vector machine, decision tree, random forest, neural network, adaptive boosting, naive Bayes, quadratic discriminant analysis, logistic regression, and linear discriminant analysis. The input variables include both building attributes and seismic intensity measures. Since the database turns out to be strongly imbalanced, the potential influence of two preprocessing techniques is examined, namely principal component analysis and synthetic minority oversampling technique. Several metrics are considered to evaluate the performance of the resulting predictive machine learning models. Moreover, this study investigates the optimal machine learning model’s robustness against uncertainties, quantifies the importance of its features, and investigates how usability classes clustering can impact its performance. Every step of the implemented procedure is deeply explained and discussed to provide useful guidelines for similar applications.

An 18F-FDG-PET/CT-based radiomics signature for estimating malignance probability of solitary pulmonary nodule.

Some solitary pulmonary nodules (SPNs) as early manifestations of lung cancer, it is difficult to determine its nature, which brings great trouble to clinical diagnosis and treatment. Radiomics can deeply explore the essence of images and provide clinical decision support for clinicians. The purpose of our study was to explore the effect of positron emission tomography (PET) with 2-deoxy-2-[fluorine-18] fluoro-d-glucose integrated with computed tomography (CT; 18F-FDG-PET/CT) combined with radiomics for predicting probability of malignancy of SPNs. We retrospectively enrolled 190 patients with SPNs confirmed by pathology from January 2013 to December 2019 in our hospital. SPNs were benign in 69 patients and malignant in 121 patients. Patients were randomly divided into a training or testing group at a ratio of 7:3. Three-dimensional regions of interest (ROIs) were manually outlined on PET and CT images, and radiomics features were extracted. Synthetic minority oversampling technique (SMOTE) method was used to balance benign and malignant samples to a ratio of 1:1. In the training group, least absolute shrinkage and selection operator (LASSO) regression analyses and Spearman correlation analyses were used to select the strongest radiomics features. Three models including PET model, CT model, and joint model were constructed using multivariate logistic regression analysis. Receiver operating characteristic (ROC) curves, calibration curves, and decision curves were plotted to evaluate diagnostic efficiency, calibration degree, and clinical usefulness of all models in training and testing groups. The estimative effectiveness of the joint model was superior to the CT or PET model alone in the training and testing groups. For the joint model, CT model, and PET model, area under the ROC curve was 0.929, 0.819, 0.833 in the training group, and 0.844, 0.759, 0.748 in the testing group, respectively. Calibration and decision curves showed good fit and clinical usefulness for the joint model in both training and testing groups. Radiomics models constructed by combining PET and CT radiomics features are valuable for distinguishing benign and malignant SPNs. The combined effect is superior to qualitative diagnoses with CT or PET radiomics models alone.

Integrated neural network and evolutionary algorithm approach for liver fibrosis staging: Can artificial intelligence reduce patient costs?

Staging liver fibrosis is important, and liver biopsy is the gold standard diagnostic tool. We aim to design and evaluate an artificial neural network (ANN) method by taking advantage of the Teaching Learning-Based Optimization (TLBO) algorithm for the prediction of liver fibrosis stage in blood donors and hepatitis C patients. We propose a method based on a selection of machine learning classification methods including multilayer perceptron (MLP) neural network, Naive Bayesian (NB), decision tree, and deep learning. Initially, the synthetic minority oversampling technique (SMOTE) is performed to address the imbalance in the dataset. Afterward, the integration of MLP and TLBO is implemented. We propose a novel algorithm that reduces the number of required patient features to seven inputs. The accuracy of MLP using 12 features is 0.903, while that of the proposed MLP with TLBO is 0.891. Besides, the diagnostic accuracy of all methods, except the model designed with the Bayesian network, increases when the SMOTE balancer is applied. The decision tree-based deep learning methods show the highest levels of accuracy with 12 features. Interestingly, with the use of TLBO and seven features, MLP reached an accuracy rate of 0.891, which is quite satisfactory when compared with those of similar studies. The proposed model provides high diagnostic accuracy, while reducing the required number of properties from the samples. The results of our study show that the recruited algorithm of our study is more straightforward, with a smaller number of required properties and similar accuracy.

An MCDM approach for Reverse vaccinology model to predict bacterial protective antigens

Reverse vaccinology (RV) is a significant step in sensible vaccine design. In recent years, many machine learning (ML) methods have been used to improve RV prediction accuracy. However, there are still issues with prediction accuracy and programme accessibility in ML-based RV. This paper presents a supervised ML-based method to classify bacterial protective antigens (BPAgs) and identify the model(s) that consistently perform well for the training dataset. Six ML classifiers are used for testing with physiochemical features extracted from a comprehensive training dataset. Selecting the best performing model from different performance metrics (accuracy, precision, recall, F1-score, and AUC-ROC) has not been easy, because all the metrics has the same importance to predict BPAgs. To fix this issue, we propose a soft and hard ranking model based on multi-criteria decision-making (MCDM) approach for selecting the best performing ML method that classifies BPAgs. First, our proposed model uses homologous proteins (positive and negative samples) from Protegen and Uniprot databases. Second, we applied four strategies of Synthetic Minority Oversampling Technique and Edited Nearest Neighbour (SMOTE-ENN) to handle the data imbalance problem and train the model using ML methods. Third, we consider MCDM-based technique for order preference by similarity to the ideal solution (TOPSIS) method integrated with soft and hard ranking model. The entropy is used to obtain weighted evaluation criteria for ranking the models. Our experimental evaluations show that the proposed method with best performing models (Random Forest and Extreme Gradient Boosting) outperforms compared to existing open-source RV methods using benchmark datasets.

Prediction of Myocardial Infarction Using a Combined Generative Adversarial Network Model and Feature-Enhanced Loss Function

Accurate risk prediction for myocardial infarction (MI) is crucial for preventive strategies, given its significant impact on global mortality and morbidity. Here, we propose a novel deep-learning approach to enhance the prediction of incident MI cases by incorporating metabolomics alongside clinical risk factors. We utilized data from the KORA cohort, including the baseline S4 and follow-up F4 studies, consisting of 1454 participants without prior history of MI. The dataset comprised 19 clinical variables and 363 metabolites. Due to the imbalanced nature of the dataset (78 observed MI cases and 1376 non-MI individuals), we employed a generative adversarial network (GAN) model to generate new incident cases, augmenting the dataset and improving feature representation. To predict MI, we further utilized multi-layer perceptron (MLP) models in conjunction with the synthetic minority oversampling technique (SMOTE) and edited nearest neighbor (ENN) methods to address overfitting and underfitting issues, particularly when dealing with imbalanced datasets. To enhance prediction accuracy, we propose a novel GAN for feature-enhanced (GFE) loss function. The GFE loss function resulted in an approximate 2% improvement in prediction accuracy, yielding a final accuracy of 70%. Furthermore, we evaluated the contribution of each clinical variable and metabolite to the predictive model and identified the 10 most significant variables, including glucose tolerance, sex, and physical activity. This is the first study to construct a deep-learning approach for producing 7-year MI predictions using the newly proposed loss function. Our findings demonstrate the promising potential of our technique in identifying novel biomarkers for MI prediction.

Explainable artificial intelligence in transport Logistics: Risk analysis for road accidents

Automobile traffic accidents represent a significant threat to global public safety, resulting in numerous injuries and fatalities annually. This paper introduces a comprehensive, explainable artificial intelligence (XAI) artifact design, integrating accident data for utilization by diverse stakeholders and decision-makers. It proposes responsible, explanatory, and interpretable models with a systems-level taxonomy categorizing aspects of driver-related behaviors associated with varying injury severity levels, thereby contributing theoretically to explainable analytics. In the initial phase, we employed various advanced techniques such as data missing at random (MAR) with Bayesian dynamic conditional imputation for addressing missing records, synthetic minority oversampling technique for data imbalance issues, and categorical boosting (CatBoost) combined with SHapley Additive exPlanations (SHAP) for determining and analyzing the importance and dependence of risk factors on injury severity. Additionally, exploratory feature analysis was conducted to uncover hidden spatiotemporal elements influencing traffic accidents and injury severity levels. We developed several predictive models in the second phase, including eXtreme Gradient Boosting (XGBoost), random forest (RF), deep neural networks (DNN), and fine-tuned parameters. Using the SHAP approach, we employed model-agnostic interpretation techniques to separate explanations from models. In the final phase, we provided an analysis and summary of the system-level taxonomy across feature categories. This involved classifying crash data into high-level causal factors using aggregate SHAP scores, illustrating how each risk factor contributes to different injury severity levels.

Feasibility of Principal Component Analysis for Multi-Class Earthquake Prediction Machine Learning Model Utilizing Geomagnetic Field Data

Geomagnetic field data have been found to contain earthquake (EQ) precursory signals; however, analyzing this high-resolution, imbalanced data presents challenges when implementing machine learning (ML). This study explored feasibility of principal component analyses (PCA) for reducing the dimensionality of global geomagnetic field data to improve the accuracy of EQ predictive models. Multi-class ML models capable of predicting EQ intensity in terms of the Mercalli Intensity Scale were developed. Ensemble and Support Vector Machine (SVM) models, known for their robustness and capabilities in handling complex relationships, were trained, while a Synthetic Minority Oversampling Technique (SMOTE) was employed to address the imbalanced EQ data. Both models were trained on PCA-extracted features from the balanced dataset, resulting in reasonable model performance. The ensemble model outperformed the SVM model in various aspects, including accuracy (77.50% vs. 75.88%), specificity (96.79% vs. 96.55%), F1-score (77.05% vs. 76.16%), and Matthew Correlation Coefficient (73.88% vs. 73.11%). These findings suggest the potential of a PCA-based ML model for more reliable EQ prediction.

Wind Shear and Aircraft Aborted Landings: A Deep Learning Perspective for Prediction and Analysis

In civil aviation, severe weather conditions such as strong wind shear, crosswinds, and thunderstorms near airport runways often compel pilots to abort landings to ensure flight safety. While aborted landings due to wind shear are not common, they occur under specific environmental and situational circumstances. This research aims to accurately predict aircraft aborted landings using three advanced deep learning techniques: the conventional deep neural network (DNN), the deep and cross network (DCN), and the wide and deep network (WDN). These models are supplemented by various data augmentation methods, including the Synthetic Minority Over-Sampling Technique (SMOTE), KMeans-SMOTE, and Borderline-SMOTE, to correct the imbalance in pilot report data. Bayesian optimization was utilized to fine-tune the models for optimal predictive accuracy. The effectiveness of these models was assessed through metrics including sensitivity, precision, F1-score, and the Matthew Correlation Coefficient. The Shapley Additive Explanations (SHAP) algorithm was then applied to the most effective models to interpret their results and identify key factors, revealing that the intensity of wind shear, specific runways like 07R, and the vertical distance of wind shear from the runway (within 700 feet above runway level) were significant factors. The results of this research provide valuable insights to civil aviation experts, potentially revolutionizing safety protocols for managing aborted landings under adverse weather conditions, thereby improving overall airport efficiency and safety.

Performance of Random Oversampling, Random Undersampling, and SMOTE-NC Methods in Handling Imbalanced Class in Classification Models

One common challenge in classification modeling is the existence of imbalanced classes within the data. If the analysis continues with imbalanced classes, it is probable that the result will demonstrate inadequate performance when forecasting new data. Various approaches exist to rectify this class imbalance issue, such as random oversampling, random undersampling, and the Synthetic Minority Over-sampling Technique for Nominal and Continuous (SMOTE-NC). Each of these methods encompasses distinct techniques aimed at achieving balanced class distribution within the dataset. Comparison of classification performance on imbalanced classes handled by these three methods has never been carried out in previous research. Therefore, this study undertakes an evaluation of classification models (specifically Gradient Boosting, Random Forest, and Extremely Randomized Trees) in the context of imbalanced class data. The results of this research show that the random undersampling method used to balance the class distribution has the best performance on two classification models (Random Forest and Gradient Boosted Tree).

Interpretable knowledge-guided framework for modeling reservoir water-sensitivity damage based on Light Gradient Boosting Machine using Bayesian optimization and hybrid feature mining

Reservoir water sensitivity damage significantly contributes to production declines in low-permeability oil and gas fields. An accurate and rapid assessment of water sensitivity is essential for effective mitigation or prevention strategies. Facing the intricate challenge of predicting high-dimensional water sensitivity damage, this study leverages trends in intelligent drilling and completion technologies. It adopts a Knowledge-guided, Bayesian Optimization-enhanced Light Gradient Boosting Machine (KBO-LightGBM) for modeling, augmented by Multiple Imputation by Chained Equations and Synthetic Minority Over-sampling Technique (MICE-SMOTE) to address missing and unbalanced data issues in oil and gas fields, thereby enhancing the scientific efficacy of data processing. The framework's precision and practicality were confirmed using data from 270 natural core samples across 15 oil fields. Findings include a correlation coefficient of 0.9679 on the test set, a root mean square error of 3.4797, and a mean absolute percentage error of 4.0936%. Interpretability analysis identified formation water mineralization, burial depth, feldspar content, and initial porosity as the predominant factors affecting water sensitivity. This research distinguishes itself with a broader dataset, covering 15 parameters of formation fluids and rock components. Weighting factor αDK and scale βDK were designed to integrate empirical correlations into LightGBM's loss function, theoretically mitigating model overfitting. Hence, the intelligent framework proposed herein accurately predicts reservoir water sensitivity damage, aiding in the development of reservoir damage control strategies.